{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8833c267",
   "metadata": {},
   "source": [
    "## 实验二：搭建VGG16神经网络实现图像分类"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e56809e",
   "metadata": {},
   "source": [
    "**实验目的**：\n",
    "    掌握卷积神经网络的设计原理，能够独立构建卷积神经网络，深入了解基本算子的正向传播及反向传播原理，能够使用 Python 语言构建 VGG16 网络模型来对给定的输入图像进行分类，能够独立编写基本算子的正向传播及反向传播代码。\n",
    "\n",
    "**实验内容**：\n",
    "* Convolution算子的正向传播及反向传播代码实现 \n",
    "* MaxPool算子的正向传播及反向传播代码实现 \n",
    "* VGG16网络搭建"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "503d5161",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入相关依赖库\n",
    "import time\n",
    "import numpy as np\n",
    "from numba import jit\n",
    "import cv2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3074aee",
   "metadata": {},
   "source": [
    "### 1.Convolution算子实现"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7fee95a",
   "metadata": {},
   "source": [
    "卷积算子的实现如下所示，其中定义了以下成员函数：\n",
    "\n",
    "* 算子初始化：需要定义卷积算子的超参数，包括输入张量的通道数$C_{in}$，输出张量的通道数$C_{out}$，卷积核的尺寸$K$，边界扩充大小$P$，卷积步长$S$。此外还需要定义输入张量的形状，用于反向传播。\n",
    "* 权重初始化：卷积算子的参数包括权重和偏置。通常使用高斯随机数来初始化权重，将偏置值均设为0。\n",
    "* 正向传播计算：根据公式进行卷积算子正向传播的计算，首先对输入张量`inputs`进行边界填充得到`inputs_pad`，在填充后的张量`inputs_pad`上滑动卷积窗口。\n",
    "* 反向传播计算：根据公式进行卷积算子反向传播的计算（因为不涉及参数更新，故忽略计算偏置的梯度）。\n",
    "* 参数加载：通过输入指定卷积算子的权重和偏置参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "217191d7",
   "metadata": {},
   "source": [
    "正向传播公式：\n",
    "$$Y(n,c,h,w) = \\sum_{k=0}^{C_{in}-1} \\sum_{i=0}^{K-1} \\sum_{j=0}^{K-1} Weight(c,k,i,j)*X_{pad}(n,k,h_{s}+i,w_{s}+j) + Bias(c) \\\\\n",
    "n \\in [0, N), c \\in [0, C_{out}), h \\in [0,H_{out}),w\\in [0,W_{out}) \\\\\n",
    "h_{s} = h*S, w_s = w*S$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3c56a05",
   "metadata": {},
   "source": [
    "反向传播公式：\n",
    "$$\\sum_{c_{in}=0}^{C_{in}-1}\\sum_{i=0}^{K-1}\\sum_{j=0}^{K-1} \\nabla_{in_{pad}}(n,c_{in},h_s+i,w_s+i) = \\sum_{c_{out}=0}^{C_{out}-1} \\nabla_{out}(n,c_{out},h,w)*Weight(c_{out},c_{in},i,j) \\\\\n",
    "n \\in [0, N), h \\in [0,H_{out}),w\\in [0,W_{out}) \\\\\n",
    "h_{s} = h*S, w_s = w*S$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "18382ab0",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ConvolutionLayer(object):\n",
    "    def __init__(self, in_channels, out_channels, kernel_size, padding=0, stride=1):\n",
    "        # 输入通道数\n",
    "        self.in_channels = in_channels\n",
    "        # 输出通道数\n",
    "        self.out_channels = out_channels\n",
    "        # 卷积核尺寸\n",
    "        self.kernel_size = kernel_size\n",
    "        # 步长\n",
    "        self.stride = stride\n",
    "        # 填充长度\n",
    "        self.padding = padding\n",
    "\n",
    "        # 卷积核权重\n",
    "        self.weight = np.random.normal(loc=0.0, scale=0.01,\n",
    "                                       size=(self.out_channels, self.in_channels,\n",
    "                                             self.kernel_size, self.kernel_size))\n",
    "        # 卷积核偏置\n",
    "        self.bias = np.zeros([self.out_channels])\n",
    "\n",
    "        # 输入张量的形状，用于反向传播\n",
    "        self.input_shape = None\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        # 记录输入张量的形状，inputs: (N,C,H,W)\n",
    "        self.input_shape = inputs.shape\n",
    "        batch, channel, height, width = inputs.shape\n",
    "\n",
    "        # 获取输入张量填充后的宽高\n",
    "        pad_height = height + self.padding * 2\n",
    "        pad_width = width + self.padding * 2\n",
    "\n",
    "        # 将输入张量进行填充\n",
    "        inputs_pad = np.zeros((batch, channel, pad_height, pad_width), dtype=inputs.dtype)\n",
    "        inputs_pad[:, :, self.padding:height + self.padding, self.padding:width + self.padding] = inputs\n",
    "\n",
    "        # 获取输出张量的宽高，并构建输出张量\n",
    "        out_height = int((pad_height - self.kernel_size) / self.stride + 1)\n",
    "        out_width = int((pad_width - self.kernel_size) / self.stride + 1)\n",
    "        outputs = np.zeros((batch, self.out_channels, out_height, out_width), dtype=inputs.dtype)\n",
    "        \n",
    "        # 正向传播\n",
    "        outputs = self._conv(inputs_pad, outputs, self.weight, self.bias, self.kernel_size, self.stride)\n",
    "        return outputs\n",
    "\n",
    "    def backward(self, out_grad):\n",
    "        # 获得输入张量，填充后输入张量，输出张量的形状\n",
    "        batch, channel, height, width = self.input_shape\n",
    "        _, out_channel, out_height, out_width = out_grad.shape\n",
    "        pad_height = height + self.padding * 2\n",
    "        pad_width = width + self.padding * 2\n",
    "\n",
    "        # 构建填充输入张量的梯度\n",
    "        in_grad = np.zeros((batch, channel, pad_height, pad_width))\n",
    "\n",
    "        # 反向传播\n",
    "        in_grad = self._conv_back(out_grad, in_grad, self.weight, self.kernel_size, self.stride)\n",
    "        \n",
    "        # 返回输入张量梯度\n",
    "        in_grad = in_grad[:, :, self.padding:height + self.padding, self.padding:width + self.padding]\n",
    "        return in_grad\n",
    "\n",
    "    def load_params(self, weight, bias):\n",
    "        assert self.weight.shape == weight.shape\n",
    "        assert self.bias.shape == bias.shape\n",
    "        self.weight = weight\n",
    "        self.bias = bias\n",
    "\n",
    "    @staticmethod\n",
    "    @jit(nopython=True)     # 可以将python函数编译为机器代码的JIT编译器，可以极大的加速for循环的运行速度\n",
    "    def _conv(inputs_pad, outputs, weight, bias, kernel_size, stride):\n",
    "        # TODO：根据公式编写下列代码 请用for循环实现\n",
    "        in_channels = inputs_pad.shape[1]\n",
    "        batch, out_channels, out_height, out_width = outputs.shape\n",
    "        for n in range(batch):\n",
    "            for c in range(out_channels):\n",
    "                for h in range(out_height):\n",
    "                    for w in range(out_width):\n",
    "                        hs, ws = h * stride, w * stride\n",
    "                        val = 0\n",
    "                        for k in range(in_channels):\n",
    "                            for i in range(kernel_size):\n",
    "                                for j in range(kernel_size):\n",
    "                                    val += weight[c, k, i, j] * inputs_pad[n, k, hs+i, ws+j]\n",
    "                        val += bias[c]\n",
    "                        outputs[n, c, h, w] = val\n",
    "        return outputs\n",
    "\n",
    "    @staticmethod\n",
    "    @jit(nopython=True)\n",
    "    def _conv_back(out_grad, in_grad, weight, kernel_size, stride):\n",
    "        # TODO：根据公式编写下列代码 请用for循环实现\n",
    "        in_channels = in_grad.shape[1]\n",
    "        batch, out_channel, out_height, out_width = out_grad.shape\n",
    "        for n in range(batch):\n",
    "            for h in range(out_height):\n",
    "                for w in range(out_width):\n",
    "                    hs, ws = h * stride, w * stride\n",
    "                    for c_in in range(in_channels):\n",
    "                        for i in range(kernel_size):\n",
    "                            for j in range(kernel_size):\n",
    "                                val = 0\n",
    "                                for c_out in range(out_channel):\n",
    "                                    val += out_grad[n, c_out, h, w] * weight[c_out, c_in, i, j]\n",
    "                                in_grad[n, c_in, hs + i, ws + j] += val\n",
    "        return in_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10e2c0d6",
   "metadata": {},
   "source": [
    "### 2.MaxPool算子实现"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "add42821",
   "metadata": {},
   "source": [
    "最大池化算子的实现如下所示，其中定义了以下成员函数：\n",
    "\n",
    "* 算子初始化：需要定义最大池化算子的超参数，包括池化核的尺寸$K$，池化步长$S$。此外初始化了用于反向传播的池化索引，输入张量的形状和输出张量的形状。\n",
    "* 正向传播计算：根据公式进行池化算子正向传播的计算。\n",
    "* 反向传播计算：根据公式进行池化算子反向传播的计算。在正向传播时，已经记录了池化索引，在反向传播时，只需将池化索引映射回输入张量的位置，将梯度带过去即可，其余位置置为0。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b109924e",
   "metadata": {},
   "source": [
    "正向传播公式：\n",
    "$$Y(n,c,h,w) = \\mathop{max}\\limits_{m=0,..K-1} \\space \\mathop{max}\\limits_{n=0,..K-1} X(n, c, h_s+m, w_s+n)\\\\\n",
    "n \\in [0, N), c \\in [0, C), h \\in [0,H_{out}),w\\in [0,W_{out}) \\\\\n",
    "h_{s} = h*S, w_s = w*S$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "237e8589",
   "metadata": {},
   "source": [
    "反向传播公式：\n",
    "$$\\nabla_{in}(n,c,h_s:h_s+K,w_s:w_s+K)[i_{index},j_{index}]=\\nabla_{out}(n,c,h,w)\\\\\n",
    "n \\in [0, N),c \\in [0, C), h \\in [0,H_{out}),w\\in [0,W_{out}) \\\\\n",
    "h_{s} = h*S, w_s = w*S \\\\\n",
    "$$\n",
    "其中 $i_{index}$ 和 $j_{index}$ 在正向传播中记录下的索引值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4319a8b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "class MaxPoolLayer(object):\n",
    "    def __init__(self, kernel_size=2, stride=2):\n",
    "        # 池化核大小\n",
    "        self.kernel_size = kernel_size\n",
    "        # 步长\n",
    "        self.stride = stride\n",
    "        # 池化索引，用于反向传播\n",
    "        self.argidx = None\n",
    "        # 输入张量形状\n",
    "        self.input_shape = None\n",
    "        # 输出张量形状\n",
    "        self.output_shape = None\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        # inputs: (N,C,H,W)\n",
    "        batch, channel, height, width = inputs.shape\n",
    "\n",
    "        # 获取输出张量的宽高，并构建输出张量\n",
    "        out_height = int((height - self.kernel_size) / self.stride + 1)\n",
    "        out_width = int((width - self.kernel_size) / self.stride + 1)\n",
    "        outputs = np.zeros((batch, channel, out_height, out_width), dtype=inputs.dtype)\n",
    "\n",
    "        # 记录输入张量和输出张量的形状，并初始化池化索引\n",
    "        self.input_shape = inputs.shape\n",
    "        self.output_shape = outputs.shape\n",
    "        self.argidx = np.zeros_like(outputs, dtype=np.int32)\n",
    "\n",
    "        # 正向传播\n",
    "        outputs, self.argidx = self._pool(outputs, inputs, self.argidx, self.kernel_size, self.stride)\n",
    "        return outputs\n",
    "\n",
    "    def backward(self, out_grad):\n",
    "        # 构建输入梯度\n",
    "        in_grad = np.zeros(self.input_shape)\n",
    "\n",
    "        # 反向传播\n",
    "        in_grad = self._pool_back(out_grad, in_grad , self.argidx, self.kernel_size, self.stride)\n",
    "        return in_grad\n",
    "\n",
    "    @staticmethod\n",
    "    @jit(nopython=True)\n",
    "    def _pool(outputs, inputs, argidx, kernel_size, stride):\n",
    "        # TODO：根据公式编写下列代码 请用for循环实现\n",
    "        batch, channel, out_height, out_width = outputs.shape\n",
    "        for n in range(batch):\n",
    "            for c in range(channel):\n",
    "                for h in range(out_height):\n",
    "                    for w in range(out_width):\n",
    "                        hs, ws = h*stride, w*stride\n",
    "                        vector = inputs[n, c, hs:hs+kernel_size, ws:ws+kernel_size]\n",
    "                        max_value = vector[0][0]\n",
    "                        for i in range(kernel_size):\n",
    "                            for j in range(kernel_size):\n",
    "                                if vector[i, j] > max_value:\n",
    "                                    max_value = vector[i, j]\n",
    "                                    # 记录当前索引\n",
    "                                    argidx[n, c, h, w] = i * kernel_size + j\n",
    "                        outputs[n, c, h, w] = max_value\n",
    "        return outputs, argidx\n",
    "\n",
    "    @staticmethod\n",
    "    @jit(nopython=True)\n",
    "    def _pool_back(out_grad, in_grad, argidx, kernel_size, stride):\n",
    "        # TODO：根据公式编写下列代码 请用for循环实现\n",
    "        batch, channel, out_height, out_width = out_grad.shape\n",
    "        for n in range(batch):\n",
    "            for c in range(channel):\n",
    "                for h in range(out_height):\n",
    "                    for w in range(out_width):\n",
    "                        hs, ws = h*stride, w*stride\n",
    "                        # 将索引逆向转换至卷积核位置\n",
    "                        i = argidx[n, c, h, w] // kernel_size\n",
    "                        j = argidx[n, c, h, w] % kernel_size\n",
    "                        in_grad[n, c, hs: hs+kernel_size, ws: ws+kernel_size][i, j] = out_grad[n, c, h, w]\n",
    "\n",
    "        return in_grad\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fbc19be",
   "metadata": {},
   "source": [
    "### 3. 扁平化算子实现"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2ceba7b",
   "metadata": {},
   "source": [
    "扁平化算子的实现如下所示，其中定义了以下成员函数：\n",
    "\n",
    "* 正向传播计算：进行Flatten算子正向传播的计算。将四维张量$(N,C,H,W)$，扁平化至二维$(N,C*H*W)$\n",
    "* 反向传播计算：进行Flatten算子反向传播的计算。将二维梯度$(N,C*H*W)$映射回四维梯度$(N,C,H,W)$即可。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d1d88005",
   "metadata": {},
   "outputs": [],
   "source": [
    "class FlattenLayer(object):\n",
    "    def __init__(self):\n",
    "        self.input_shape = None\n",
    "\n",
    "    def forward(self, inputs):\n",
    "        # inputs: (N,C,H,W) -> (N, C*H*W)\n",
    "        self.input_shape = inputs.shape\n",
    "        batch, channel, height, width = inputs.shape\n",
    "        return inputs.reshape((batch, channel * height * width))\n",
    "\n",
    "    def backward(self, out_grad):\n",
    "        return out_grad.reshape(self.input_shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "584b9ea2",
   "metadata": {},
   "source": [
    "### 4.搭建VGG16网络"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "1d3316b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 导入算子\n",
    "from layer import ReluLayer, FullyConnectLayer, CrossEntropy\n",
    "\n",
    "class VGG16(object):\n",
    "    def __init__(self, num_classes=4):\n",
    "        # TODO 根据网络图搭建VGG16模型\n",
    "        self.layer1_conv1 = ConvolutionLayer(in_channels=3, out_channels=64, kernel_size=3, padding=1)\n",
    "        self.layer1_relu1 = ReluLayer()\n",
    "        self.layer1_conv2 = ConvolutionLayer(in_channels=64, out_channels=64, kernel_size=3, padding=1)\n",
    "        self.layer1_relu2 = ReluLayer()\n",
    "        self.layer1_maxpool = MaxPoolLayer(kernel_size=2, stride=2)\n",
    "\n",
    "        self.layer2_conv1 = ConvolutionLayer(in_channels=64, out_channels=128, kernel_size=3, padding=1)\n",
    "        self.layer2_relu1 = ReluLayer()\n",
    "        self.layer2_conv2 = ConvolutionLayer(in_channels=128, out_channels=128, kernel_size=3, padding=1)\n",
    "        self.layer2_relu2 = ReluLayer()\n",
    "        self.layer2_maxpool = MaxPoolLayer(kernel_size=2, stride=2)\n",
    "\n",
    "        self.layer3_conv1 = ConvolutionLayer(in_channels=128, out_channels=256, kernel_size=3, padding=1)\n",
    "        self.layer3_relu1 = ReluLayer()\n",
    "        self.layer3_conv2 = ConvolutionLayer(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n",
    "        self.layer3_relu2 = ReluLayer()\n",
    "        self.layer3_conv3 = ConvolutionLayer(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n",
    "        self.layer3_relu3 = ReluLayer()\n",
    "        self.layer3_maxpool = MaxPoolLayer(kernel_size=2, stride=2)\n",
    "\n",
    "        self.layer4_conv1 = ConvolutionLayer(in_channels=256, out_channels=512, kernel_size=3, padding=1)\n",
    "        self.layer4_relu1 = ReluLayer()\n",
    "        self.layer4_conv2 = ConvolutionLayer(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
    "        self.layer4_relu2 = ReluLayer()\n",
    "        self.layer4_conv3 = ConvolutionLayer(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
    "        self.layer4_relu3 = ReluLayer()\n",
    "        self.layer4_maxpool = MaxPoolLayer(kernel_size=2, stride=2)\n",
    "\n",
    "        self.layer5_conv1 = ConvolutionLayer(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
    "        self.layer5_relu1 = ReluLayer()\n",
    "        self.layer5_conv2 = ConvolutionLayer(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
    "        self.layer5_relu2 = ReluLayer()\n",
    "        self.layer5_conv3 = ConvolutionLayer(in_channels=512, out_channels=512, kernel_size=3, padding=1)\n",
    "        self.layer5_relu3 = ReluLayer()\n",
    "        self.layer5_maxpool = MaxPoolLayer(kernel_size=2, stride=2)\n",
    "\n",
    "        self.flatten = FlattenLayer()\n",
    "        self.fullyconnect1 = FullyConnectLayer(in_features=512 * 7 * 7, out_features=4096)\n",
    "        self.relu_1 = ReluLayer()\n",
    "        self.fullyconnect2 = FullyConnectLayer(in_features=4096, out_features=4096)\n",
    "        self.relu_2 = ReluLayer()\n",
    "        self.fullyconnect3 = FullyConnectLayer(in_features=4096, out_features=num_classes)\n",
    "\n",
    "        self.graph_layers = None\n",
    "        self.create_graph()\n",
    "\n",
    "    def create_graph(self):\n",
    "        self.graph_layers = {\n",
    "            'layer1_conv1': self.layer1_conv1, 'layer1_relu1': self.layer1_relu1,\n",
    "            'layer1_conv2': self.layer1_conv2, 'layer1_relu2': self.layer1_relu2,\n",
    "            'layer1_maxpool': self.layer1_maxpool,\n",
    "\n",
    "            'layer2_conv1': self.layer2_conv1, 'layer2_relu1': self.layer2_relu1,\n",
    "            'layer2_conv2': self.layer2_conv2, 'layer2_relu2': self.layer2_relu2,\n",
    "            'layer2_maxpool': self.layer2_maxpool,\n",
    "\n",
    "            'layer3_conv1': self.layer3_conv1, 'layer3_relu1': self.layer3_relu1,\n",
    "            'layer3_conv2': self.layer3_conv2, 'layer3_relu2': self.layer3_relu2,\n",
    "            'layer3_conv3': self.layer3_conv3, 'layer3_relu3': self.layer3_relu3,\n",
    "            'layer3_maxpool': self.layer3_maxpool,\n",
    "\n",
    "            'layer4_conv1': self.layer4_conv1, 'layer4_relu1': self.layer4_relu1,\n",
    "            'layer4_conv2': self.layer4_conv2, 'layer4_relu2': self.layer4_relu2,\n",
    "            'layer4_conv3': self.layer4_conv3, 'layer4_relu3': self.layer4_relu3,\n",
    "            'layer4_maxpool': self.layer4_maxpool,\n",
    "\n",
    "            'layer5_conv1': self.layer5_conv1, 'layer5_relu1': self.layer5_relu1,\n",
    "            'layer5_conv2': self.layer5_conv2, 'layer5_relu2': self.layer5_relu2,\n",
    "            'layer5_conv3': self.layer5_conv3, 'layer5_relu3': self.layer5_relu3,\n",
    "            'layer5_maxpool': self.layer5_maxpool,\n",
    "\n",
    "            'flatten': self.flatten,\n",
    "            'fullyconnect1': self.fullyconnect1, 'relu1': self.relu_1,\n",
    "            'fullyconnect2': self.fullyconnect2, 'relu2': self.relu_2,\n",
    "            'fullyconnect3': self.fullyconnect3,\n",
    "        }\n",
    "\n",
    "    def forward(self, x):\n",
    "        for name in self.graph_layers.keys():\n",
    "            # 正向传播的同时，打印均值和总和，用于核对执行过程是否正确\n",
    "            print(f'forward: {name}: {x.mean()} {x.sum()}')\n",
    "            x = self.graph_layers[name].forward(x)\n",
    "        return x\n",
    "\n",
    "    def backward(self, grad):\n",
    "        for name in reversed(list(self.graph_layers.keys())):\n",
    "            # 反向传播的同时，打印均值和总和，用于核对执行过程是否正确\n",
    "            print(f'backward: {name}: {grad.mean()} {grad.sum()}')\n",
    "            grad = self.graph_layers[name].backward(grad)\n",
    "        return grad\n",
    "\n",
    "    def resume_weights(self, ckpt):\n",
    "        for name, params in ckpt.items():\n",
    "            self.graph_layers[name].load_params(params['weight'], params['bias'])\n",
    "        print('reloaded success')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "122d7140",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "reloaded success\n",
      "forward: layer1_conv1: -0.06873162090778351 -10346.033203125\n",
      "forward: layer1_relu1: -0.05778738483786583 -185570.546875\n",
      "forward: layer1_conv2: 0.28507915139198303 915464.4375\n",
      "forward: layer1_relu2: -0.013939479365944862 -44763.34765625\n",
      "forward: layer1_maxpool: 0.33006376028060913 1059921.875\n",
      "forward: layer2_conv1: 0.4298982322216034 345129.1875\n",
      "forward: layer2_relu1: 0.017751257866621017 28501.98828125\n",
      "forward: layer2_conv2: 0.3358146846294403 539194.8125\n",
      "forward: layer2_relu2: 0.029406482353806496 47215.98828125\n",
      "forward: layer2_maxpool: 0.34632712602615356 556073.9375\n",
      "forward: layer3_conv1: 0.5311101675033569 213191.875\n",
      "forward: layer3_relu1: 0.003264831844717264 2621.059326171875\n",
      "forward: layer3_conv2: 0.3423604369163513 274852.4375\n",
      "forward: layer3_relu2: 0.018941137939691544 15206.248046875\n",
      "forward: layer3_conv3: 0.3454575836658478 277338.875\n",
      "forward: layer3_relu3: -0.013854059390723705 -11122.2607421875\n",
      "forward: layer3_maxpool: 0.32738643884658813 262831.0625\n",
      "forward: layer4_conv1: 0.596550703048706 119730.109375\n",
      "forward: layer4_relu1: 0.0014814476016908884 594.6649169921875\n",
      "forward: layer4_conv2: 0.33410972356796265 134114.3125\n",
      "forward: layer4_relu2: 0.0025864504277706146 1038.221923828125\n",
      "forward: layer4_conv3: 0.3303680717945099 132612.390625\n",
      "forward: layer4_relu3: 0.037846896797418594 15192.046875\n",
      "forward: layer4_maxpool: 0.3491385281085968 140147.0\n",
      "forward: layer5_conv1: 0.695088267326355 69753.5\n",
      "forward: layer5_relu1: 0.01296419557183981 1300.98291015625\n",
      "forward: layer5_conv2: 0.35738039016723633 35863.8359375\n",
      "forward: layer5_relu2: -0.03126288950443268 -3137.29345703125\n",
      "forward: layer5_conv3: 0.35006460547447205 35129.68359375\n",
      "forward: layer5_relu3: -0.007918775081634521 -794.6649169921875\n",
      "forward: layer5_maxpool: 0.36700019240379333 36829.203125\n",
      "forward: flatten: 0.7064959406852722 17724.5703125\n",
      "forward: fullyconnect1: 0.7064959406852722 17724.5703125\n",
      "forward: relu1: 0.09846815466880798 403.3255615234375\n",
      "forward: fullyconnect2: 0.3777786195278168 1547.3812255859375\n",
      "forward: relu2: 0.050107765942811966 205.2414093017578\n",
      "forward: fullyconnect3: 0.2082405686378479 852.953369140625\n",
      "forward outputs: [[-2.7988272  0.4677529 -2.3161106  5.406888 ]]\n",
      "predict class: tulips\n",
      "loss: 4.9469804763793945\n",
      "backward: fullyconnect3: 7.865310180932283e-09 3.1461240723729134e-08\n",
      "backward: relu2: 6.210785664333555e-05 0.2543937808111024\n",
      "backward: fullyconnect2: 0.0012413157707408605 5.084429396954564\n",
      "backward: relu1: 0.00024674972706713506 1.0106868820669852\n",
      "backward: fullyconnect1: 0.000619498656067536 2.5374664952526276\n",
      "backward: flatten: 8.395934425359038e-05 2.1063720286340755\n",
      "backward: layer5_maxpool: 8.395934425359038e-05 2.1063720286340755\n",
      "backward: layer5_relu3: 2.098983606339759e-05 2.106372028634075\n",
      "backward: layer5_conv3: 2.9413559685998462e-05 2.9517095416093175\n",
      "backward: layer5_relu2: 8.840234643007457e-06 0.8871352268950844\n",
      "backward: layer5_conv2: 3.344091892854457e-05 3.3558630963173046\n",
      "backward: layer5_relu1: 2.167175697222552e-05 2.1748041556767754\n",
      "backward: layer5_conv1: 4.0473638719889604e-05 4.0616105928183615\n",
      "backward: layer4_maxpool: 3.531487226764977e-05 3.54391806180319\n",
      "backward: layer4_relu3: 8.828718066912442e-06 3.5439180618031894\n",
      "backward: layer4_conv3: 1.2349069386762725e-05 4.957015244401652\n",
      "backward: layer4_relu2: 7.974776117380638e-07 0.32011389317255273\n",
      "backward: layer4_conv2: 1.3892953209484716e-05 5.576742561912841\n",
      "backward: layer4_relu1: -7.827718958464306e-06 -3.1421090116792403\n",
      "backward: layer4_conv1: 9.923590321488867e-06 3.9834085437682027\n",
      "backward: layer3_maxpool: -7.271325844132415e-06 -1.4593841822207523\n",
      "backward: layer3_relu3: -1.817831461033104e-06 -1.4593841822207525\n",
      "backward: layer3_conv3: 1.2824905740728073e-06 1.0296039527148348\n",
      "backward: layer3_relu2: 1.1030134127270366e-05 8.855168159518685\n",
      "backward: layer3_conv2: 1.030884855550387e-05 8.276108561935395\n",
      "backward: layer3_relu1: 8.370184590148615e-06 6.71971811192475\n",
      "backward: layer3_conv1: 1.2672711611982089e-05 10.173855645485013\n",
      "backward: layer2_maxpool: -4.749556799829168e-06 -1.9065100959058268\n",
      "backward: layer2_relu2: -1.1873891999572916e-06 -1.906510095905826\n",
      "backward: layer2_conv2: 2.6177596383650433e-06 4.203158643667341\n",
      "backward: layer2_relu1: 1.5477010926947416e-07 0.24850384008656434\n",
      "backward: layer2_conv1: 4.2199326814203514e-06 6.775658951134322\n",
      "backward: layer1_maxpool: -8.022843319657e-06 -6.440866982513754\n",
      "backward: layer1_relu2: -2.005710829914249e-06 -6.440866982513752\n",
      "backward: layer1_conv2: -3.6525756662366717e-07 -1.172938474426184\n",
      "backward: layer1_relu1: 9.582066432380779e-08 0.3077054497991283\n",
      "backward: layer1_conv1: 4.2874589162936326e-07 1.3768162469372756\n",
      "current task cost time: 88.6858229637146\n"
     ]
    }
   ],
   "source": [
    "# 导入预处理函数\n",
    "from main import process_image\n",
    "\n",
    "# 分类类别\n",
    "CLASSES = ('daisy', 'roses', 'sunflowers', 'tulips')\n",
    "\n",
    "# 网络初始化、加载权重参数\n",
    "model = VGG16(4)\n",
    "ckpt = np.load('./file/vgg16_ckpt.npy', allow_pickle=True).item()\n",
    "model.resume_weights(ckpt)\n",
    "\n",
    "start_time = time.time()\n",
    "\n",
    "# 输入图片预处理\n",
    "image_path = './file/tulips_demo.jpg'\n",
    "tensor = process_image(image_path)\n",
    "\n",
    "# 模型正向传播\n",
    "outputs = model.forward(tensor)\n",
    "print(f'forward outputs: {outputs}')\n",
    "pred = int(np.argmax(outputs))\n",
    "print(f'predict class: {CLASSES[pred]}')\n",
    "\n",
    "# 计算loss\n",
    "label = np.array([1, ])\n",
    "loss_func = CrossEntropy()\n",
    "loss = loss_func.forward(outputs, label)\n",
    "print(f'loss: {loss}')\n",
    "\n",
    "# 反向传播\n",
    "grad = loss_func.backward()\n",
    "grad = model.backward(grad)\n",
    "\n",
    "end_time = time.time()\n",
    "print(f'current task cost time: {end_time - start_time}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e7f18ca",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7.5 64-bit",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  },
  "vscode": {
   "interpreter": {
    "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
